Relation-Aware Language-Graph Transformer for Question Answering

Question Answering (QA) is a task that entails reasoning over natural language contexts, and many relevant works augment models (LMs) with graph neural networks (GNNs) to encode the Knowledge Graph (KG) information. However, most existing GNN-based modules for QA do not take advantage of rich relational information KGs depend on limited interaction between LM KG. To address these issues, we propose Transformer (QAT), which designed jointly reason graphs respect entity relations in unified manner. Specifically, QAT constructs Meta-Path tokens, learn relation-centric embeddings based diverse structural semantic relations. Then, our Relation-Aware Self-Attention module comprehensively integrates different modalities via Cross-Modal Relative Position Bias, guides exchange entities modalities. We validate effectiveness commonsense question answering datasets like CommonsenseQA OpenBookQA, medical dataset, MedQA-USMLE. On all datasets, method achieves state-of-the-art performance. Our code available at http://github.com/mlvlab/QAT.